<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Support Vector Machines for Non-Linearly Separable Data</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d9/df8/tutorial_root.html">OpenCV Tutorials</a></li><li class="navelem"><a class="el" href="../../d3/dd5/tutorial_table_of_content_other.html">Other tutorials (ml, objdetect, photo, stitching, video)</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Support Vector Machines for Non-Linearly Separable Data </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p><b>Prev Tutorial:</b> <a class="el" href="../../d1/d73/tutorial_introduction_to_svm.html">Introduction to Support Vector Machines</a></p>
<p><b>Next Tutorial:</b> <a class="el" href="../../d1/dee/tutorial_introduction_to_pca.html">Introduction to Principal Component Analysis (PCA)</a></p>
<table class="doxtable">
<tr>
<th align="right"></th><th align="left"></th></tr>
<tr>
<td align="right">Original author </td><td align="left">Fernando Iglesias García </td></tr>
<tr>
<td align="right">Compatibility </td><td align="left">OpenCV &gt;= 3.0 </td></tr>
</table>
<h2>Goal </h2>
<p>In this tutorial you will learn how to:</p>
<ul>
<li>Define the optimization problem for SVMs when it is not possible to separate linearly the training data.</li>
<li>How to configure the parameters to adapt your SVM for this class of problems.</li>
</ul>
<h2>Motivation </h2>
<p>Why is it interesting to extend the SVM optimization problem in order to handle non-linearly separable training data? Most of the applications in which SVMs are used in computer vision require a more powerful tool than a simple linear classifier. This stems from the fact that in these tasks <b>the training data can be rarely separated using an hyperplane</b>.</p>
<p>Consider one of these tasks, for example, face detection. The training data in this case is composed by a set of images that are faces and another set of images that are non-faces (<em>every other thing in the world except from faces</em>). This training data is too complex so as to find a representation of each sample (<em>feature vector</em>) that could make the whole set of faces linearly separable from the whole set of non-faces.</p>
<h2>Extension of the Optimization Problem </h2>
<p>Remember that using SVMs we obtain a separating hyperplane. Therefore, since the training data is now non-linearly separable, we must admit that the hyperplane found will misclassify some of the samples. This <em>misclassification</em> is a new variable in the optimization that must be taken into account. The new model has to include both the old requirement of finding the hyperplane that gives the biggest margin and the new one of generalizing the training data correctly by not allowing too many classification errors.</p>
<p>We start here from the formulation of the optimization problem of finding the hyperplane which maximizes the <b>margin</b> (this is explained in the previous tutorial (<a class="el" href="../../d1/d73/tutorial_introduction_to_svm.html">Introduction to Support Vector Machines</a>):</p>
<p class="formulaDsp">
\[\min_{\beta, \beta_{0}} L(\beta) = \frac{1}{2}||\beta||^{2} \text{ subject to } y_{i}(\beta^{T} x_{i} + \beta_{0}) \geq 1 \text{ } \forall i\]
</p>
<p>There are multiple ways in which this model can be modified so it takes into account the misclassification errors. For example, one could think of minimizing the same quantity plus a constant times the number of misclassification errors in the training data, i.e.:</p>
<p class="formulaDsp">
\[\min ||\beta||^{2} + C \text{(misclassification errors)}\]
</p>
<p>However, this one is not a very good solution since, among some other reasons, we do not distinguish between samples that are misclassified with a small distance to their appropriate decision region or samples that are not. Therefore, a better solution will take into account the <em>distance of the misclassified samples to their correct decision regions</em>, i.e.:</p>
<p class="formulaDsp">
\[\min ||\beta||^{2} + C \text{(distance of misclassified samples to their correct regions)}\]
</p>
<p>For each sample of the training data a new parameter \(\xi_{i}\) is defined. Each one of these parameters contains the distance from its corresponding training sample to their correct decision region. The following picture shows non-linearly separable training data from two classes, a separating hyperplane and the distances to their correct regions of the samples that are misclassified.</p>
<div class="image">
<img src="../../sample-errors-dist.png" alt="sample-errors-dist.png"/>
</div>
<dl class="section note"><dt>Note</dt><dd>Only the distances of the samples that are misclassified are shown in the picture. The distances of the rest of the samples are zero since they lay already in their correct decision region.</dd></dl>
<p>The red and blue lines that appear on the picture are the margins to each one of the decision regions. It is very <b>important</b> to realize that each of the \(\xi_{i}\) goes from a misclassified training sample to the margin of its appropriate region.</p>
<p>Finally, the new formulation for the optimization problem is:</p>
<p class="formulaDsp">
\[\min_{\beta, \beta_{0}} L(\beta) = ||\beta||^{2} + C \sum_{i} {\xi_{i}} \text{ subject to } y_{i}(\beta^{T} x_{i} + \beta_{0}) \geq 1 - \xi_{i} \text{ and } \xi_{i} \geq 0 \text{ } \forall i\]
</p>
<p>How should the parameter C be chosen? It is obvious that the answer to this question depends on how the training data is distributed. Although there is no general answer, it is useful to take into account these rules:</p>
<ul>
<li>Large values of C give solutions with <em>less misclassification errors</em> but a <em>smaller margin</em>. Consider that in this case it is expensive to make misclassification errors. Since the aim of the optimization is to minimize the argument, few misclassifications errors are allowed.</li>
<li>Small values of C give solutions with <em>bigger margin</em> and <em>more classification errors</em>. In this case the minimization does not consider that much the term of the sum so it focuses more on finding a hyperplane with big margin.</li>
</ul>
<h2>Source Code </h2>
<p>You may also find the source code in <code>samples/cpp/tutorial_code/ml/non_linear_svms</code> folder of the OpenCV source library or <a href="https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/ml/non_linear_svms/non_linear_svms.cpp">download it from here</a>.</p>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'><ul>
<li><b>Downloadable code</b>: Click <a href="https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/ml/non_linear_svms/non_linear_svms.cpp">here</a></li>
<li><b>Code at glance:</b> <div class="fragment"><div class="line"><span class="preprocessor">#include &lt;iostream&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d0/d9c/core_2include_2opencv2_2core_8hpp.html">opencv2/core.hpp</a>&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d1/d4f/imgproc_2include_2opencv2_2imgproc_8hpp.html">opencv2/imgproc.hpp</a>&gt;</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d6/d87/imgcodecs_8hpp.html">opencv2/imgcodecs.hpp</a>&quot;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d4/dd5/highgui_8hpp.html">opencv2/highgui.hpp</a>&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d3/d29/ml_8hpp.html">opencv2/ml.hpp</a>&gt;</span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="../../d2/d75/namespacecv.html">cv</a>;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="../../d8/df1/namespacecv_1_1ml.html">cv::ml</a>;</div><div class="line"><span class="keyword">using namespace </span>std;</div><div class="line"></div><div class="line"><span class="keyword">static</span> <span class="keywordtype">void</span> help()</div><div class="line">{</div><div class="line">    cout&lt;&lt; <span class="stringliteral">&quot;\n--------------------------------------------------------------------------&quot;</span> &lt;&lt; endl</div><div class="line">        &lt;&lt; <span class="stringliteral">&quot;This program shows Support Vector Machines for Non-Linearly Separable Data. &quot;</span> &lt;&lt; endl</div><div class="line">        &lt;&lt; <span class="stringliteral">&quot;--------------------------------------------------------------------------&quot;</span>   &lt;&lt; endl</div><div class="line">        &lt;&lt; endl;</div><div class="line">}</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main()</div><div class="line">{</div><div class="line">    help();</div><div class="line"></div><div class="line">    <span class="keyword">const</span> <span class="keywordtype">int</span> NTRAINING_SAMPLES = 100;         <span class="comment">// Number of training samples per class</span></div><div class="line">    <span class="keyword">const</span> <span class="keywordtype">float</span> FRAC_LINEAR_SEP = 0.9f;        <span class="comment">// Fraction of samples which compose the linear separable part</span></div><div class="line"></div><div class="line">    <span class="comment">// Data for visual representation</span></div><div class="line">    <span class="keyword">const</span> <span class="keywordtype">int</span> WIDTH = 512, HEIGHT = 512;</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> I = <a class="code" href="../../d3/d63/classcv_1_1Mat.html#a0b57b6a326c8876d944d188a46e0f556">Mat::zeros</a>(HEIGHT, WIDTH, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga88c4cd9de76f678f33928ef1e3f96047">CV_8UC3</a>);</div><div class="line"></div><div class="line">    <span class="comment">//--------------------- 1. Set up training data randomly ---------------------------------------</span></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> trainData(2*NTRAINING_SAMPLES, 2, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>);</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> labels   (2*NTRAINING_SAMPLES, 1, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4067910fc388075c3ea3aa14393e83b9">CV_32S</a>);</div><div class="line"></div><div class="line">    <a class="code" href="../../d1/dd6/classcv_1_1RNG.html">RNG</a> rng(100); <span class="comment">// Random value generation class</span></div><div class="line"></div><div class="line">    <span class="comment">// Set up the linearly separable part of the training data</span></div><div class="line">    <span class="keywordtype">int</span> nLinearSamples = (int) (FRAC_LINEAR_SEP * NTRAINING_SAMPLES);</div><div class="line"></div><div class="line">    <span class="comment">// Generate random points for the class 1</span></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> trainClass = trainData.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(0, nLinearSamples);</div><div class="line">    <span class="comment">// The x coordinate of the points is in [0, 0.4)</span></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(0, 1);</div><div class="line">    rng.fill(c, <a class="code" href="../../d1/dd6/classcv_1_1RNG.html#aa1ca9b6b8a00b2d04288f9f5b5e68238af6797281458bec27565ee730ea228d4a">RNG::UNIFORM</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.4 * WIDTH));</div><div class="line">    <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">    c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(1,2);</div><div class="line">    rng.fill(c, <a class="code" href="../../d1/dd6/classcv_1_1RNG.html#aa1ca9b6b8a00b2d04288f9f5b5e68238af6797281458bec27565ee730ea228d4a">RNG::UNIFORM</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HEIGHT));</div><div class="line"></div><div class="line">    <span class="comment">// Generate random points for the class 2</span></div><div class="line">    trainClass = trainData.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(2*NTRAINING_SAMPLES-nLinearSamples, 2*NTRAINING_SAMPLES);</div><div class="line">    <span class="comment">// The x coordinate of the points is in [0.6, 1]</span></div><div class="line">    c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(0 , 1);</div><div class="line">    rng.fill(c, <a class="code" href="../../d1/dd6/classcv_1_1RNG.html#aa1ca9b6b8a00b2d04288f9f5b5e68238af6797281458bec27565ee730ea228d4a">RNG::UNIFORM</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.6*WIDTH), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(WIDTH));</div><div class="line">    <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">    c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(1,2);</div><div class="line">    rng.fill(c, <a class="code" href="../../d1/dd6/classcv_1_1RNG.html#aa1ca9b6b8a00b2d04288f9f5b5e68238af6797281458bec27565ee730ea228d4a">RNG::UNIFORM</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HEIGHT));</div><div class="line"></div><div class="line">    <span class="comment">//------------------ Set up the non-linearly separable part of the training data ---------------</span></div><div class="line"><span class="comment"></span>    <span class="comment">// Generate random points for the classes 1 and 2</span></div><div class="line">    trainClass = trainData.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(nLinearSamples, 2*NTRAINING_SAMPLES-nLinearSamples);</div><div class="line">    <span class="comment">// The x coordinate of the points is in [0.4, 0.6)</span></div><div class="line">    c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(0,1);</div><div class="line">    rng.fill(c, <a class="code" href="../../d1/dd6/classcv_1_1RNG.html#aa1ca9b6b8a00b2d04288f9f5b5e68238af6797281458bec27565ee730ea228d4a">RNG::UNIFORM</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.4*WIDTH), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.6*WIDTH));</div><div class="line">    <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">    c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(1,2);</div><div class="line">    rng.fill(c, <a class="code" href="../../d1/dd6/classcv_1_1RNG.html#aa1ca9b6b8a00b2d04288f9f5b5e68238af6797281458bec27565ee730ea228d4a">RNG::UNIFORM</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HEIGHT));</div><div class="line"></div><div class="line">    <span class="comment">//------------------------- Set up the labels for the classes ---------------------------------</span></div><div class="line">    labels.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(                0,   NTRAINING_SAMPLES).<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a0440e2a164c0b0d8462fb1e487be9876">setTo</a>(1);  <span class="comment">// Class 1</span></div><div class="line">    labels.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(NTRAINING_SAMPLES, 2*NTRAINING_SAMPLES).<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a0440e2a164c0b0d8462fb1e487be9876">setTo</a>(2);  <span class="comment">// Class 2</span></div><div class="line"></div><div class="line">    <span class="comment">//------------------------ 2. Set up the support vector machines parameters --------------------</span></div><div class="line">    cout &lt;&lt; <span class="stringliteral">&quot;Starting training process&quot;</span> &lt;&lt; endl;</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;SVM&gt;</a> svm = <a class="code" href="../../d1/d2d/classcv_1_1ml_1_1SVM.html#a2fe8b5bf897c34b8e911397b42e2cb44">SVM::create</a>();</div><div class="line">    svm-&gt;setType(<a class="code" href="../../d1/d2d/classcv_1_1ml_1_1SVM.html#ab4b93a4c42bbe213ffd9fb3832c6c44fa18157ccaec6a252b901cff6de285d608">SVM::C_SVC</a>);</div><div class="line">    svm-&gt;setC(0.1);</div><div class="line">    svm-&gt;setKernel(<a class="code" href="../../d1/d2d/classcv_1_1ml_1_1SVM.html#aad7f1aaccced3c33bb256640910a0e56ab92a19ab0c193735c3fd71f938dd87e7">SVM::LINEAR</a>);</div><div class="line">    svm-&gt;setTermCriteria(<a class="code" href="../../d9/d5d/classcv_1_1TermCriteria.html">TermCriteria</a>(<a class="code" href="../../d9/d5d/classcv_1_1TermCriteria.html#a56fecdc291ccaba8aad27d67ccf72c57a56ca2bc5cd06345060a1c1c66a8fc06e">TermCriteria::MAX_ITER</a>, (<span class="keywordtype">int</span>)1e7, 1e-6));</div><div class="line"></div><div class="line">    <span class="comment">//------------------------ 3. Train the svm ----------------------------------------------------</span></div><div class="line"><span class="comment"></span>    svm-&gt;train(trainData, <a class="code" href="../../dd/ded/group__ml.html#gga9c57a2b823008dda53d2c7f7059a8710ab8565ac2eb79152a4e3c80b0e9c9fd4c">ROW_SAMPLE</a>, labels);</div><div class="line">    cout &lt;&lt; <span class="stringliteral">&quot;Finished training process&quot;</span> &lt;&lt; endl;</div><div class="line"></div><div class="line">    <span class="comment">//------------------------ 4. Show the decision regions ----------------------------------------</span></div><div class="line"><span class="comment"></span>    <a class="code" href="../../d6/dcf/classcv_1_1Vec.html">Vec3b</a> green(0,100,0), blue(100,0,0);</div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; I.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#abed816466c45234254d25bc59c31245e">rows</a>; i++)</div><div class="line">    {</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> j = 0; j &lt; I.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa3e5a47585c9ef6a0842556739155e3e">cols</a>; j++)</div><div class="line">        {</div><div class="line">            <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> sampleMat = (<a class="code" href="../../df/dfc/classcv_1_1Mat__.html">Mat_&lt;float&gt;</a>(1,2) &lt;&lt; j, i);</div><div class="line">            <span class="keywordtype">float</span> response = svm-&gt;predict(sampleMat);</div><div class="line"></div><div class="line">            <span class="keywordflow">if</span>      (response == 1) I.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa5d20fc86d41d59e4d71ae93daee9726">at</a>&lt;<a class="code" href="../../d6/dcf/classcv_1_1Vec.html">Vec3b</a>&gt;(i,j) = green;</div><div class="line">            <span class="keywordflow">else</span> <span class="keywordflow">if</span> (response == 2) I.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa5d20fc86d41d59e4d71ae93daee9726">at</a>&lt;<a class="code" href="../../d6/dcf/classcv_1_1Vec.html">Vec3b</a>&gt;(i,j) = blue;</div><div class="line">        }</div><div class="line">    }</div><div class="line"></div><div class="line">    <span class="comment">//----------------------- 5. Show the training data --------------------------------------------</span></div><div class="line"><span class="comment"></span>    <span class="keywordtype">int</span> thick = -1;</div><div class="line">    <span class="keywordtype">float</span> px, py;</div><div class="line">    <span class="comment">// Class 1</span></div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; NTRAINING_SAMPLES; i++)</div><div class="line">    {</div><div class="line">        px = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,0);</div><div class="line">        py = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,1);</div><div class="line">        <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">circle</a>(I, <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>( (<span class="keywordtype">int</span>) px,  (<span class="keywordtype">int</span>) py ), 3, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0, 255, 0), thick);</div><div class="line">    }</div><div class="line">    <span class="comment">// Class 2</span></div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = NTRAINING_SAMPLES; i &lt;2*NTRAINING_SAMPLES; i++)</div><div class="line">    {</div><div class="line">        px = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,0);</div><div class="line">        py = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,1);</div><div class="line">        <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">circle</a>(I, <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>( (<span class="keywordtype">int</span>) px, (<span class="keywordtype">int</span>) py ), 3, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(255, 0, 0), thick);</div><div class="line">    }</div><div class="line"></div><div class="line">    <span class="comment">//------------------------- 6. Show support vectors --------------------------------------------</span></div><div class="line"><span class="comment"></span>    thick = 2;</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> sv = svm-&gt;getUncompressedSupportVectors();</div><div class="line"></div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; sv.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#abed816466c45234254d25bc59c31245e">rows</a>; i++)</div><div class="line">    {</div><div class="line">        <span class="keyword">const</span> <span class="keywordtype">float</span>* v = sv.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a13acd320291229615ef15f96ff1ff738">ptr</a>&lt;<span class="keywordtype">float</span>&gt;(i);</div><div class="line">        <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">circle</a>(I,  <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>( (<span class="keywordtype">int</span>) v[0], (<span class="keywordtype">int</span>) v[1]), 6, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(128, 128, 128), thick);</div><div class="line">    }</div><div class="line"></div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">imwrite</a>(<span class="stringliteral">&quot;result.png&quot;</span>, I);                      <span class="comment">// save the Image</span></div><div class="line">    <a class="code" href="../../d7/dfc/group__highgui.html#ga453d42fe4cb60e5723281a89973ee563">imshow</a>(<span class="stringliteral">&quot;SVM for Non-Linear Training Data&quot;</span>, I); <span class="comment">// show it to the user</span></div><div class="line">    <a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">waitKey</a>();</div><div class="line">    <span class="keywordflow">return</span> 0;</div><div class="line">}</div></div><!-- fragment -->  </div> </li>
</ul>
 <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'><ul>
<li><b>Downloadable code</b>: Click <a href="https://github.com/opencv/opencv/tree/master/samples/java/tutorial_code/ml/non_linear_svms/NonLinearSVMsDemo.java">here</a></li>
<li><b>Code at glance:</b> <div class="fragment"><div class="line"><span class="keyword">import</span> java.util.Random;</div><div class="line"></div><div class="line"><span class="keyword">import</span> org.opencv.core.Core;</div><div class="line"><span class="keyword">import</span> org.opencv.core.CvType;</div><div class="line"><span class="keyword">import</span> org.opencv.core.Mat;</div><div class="line"><span class="keyword">import</span> org.opencv.core.Point;</div><div class="line"><span class="keyword">import</span> org.opencv.core.Scalar;</div><div class="line"><span class="keyword">import</span> org.opencv.core.TermCriteria;</div><div class="line"><span class="keyword">import</span> org.opencv.highgui.HighGui;</div><div class="line"><span class="keyword">import</span> org.opencv.imgcodecs.Imgcodecs;</div><div class="line"><span class="keyword">import</span> org.opencv.imgproc.Imgproc;</div><div class="line"><span class="keyword">import</span> org.opencv.ml.Ml;</div><div class="line"><span class="keyword">import</span> org.opencv.ml.SVM;</div><div class="line"></div><div class="line"><span class="keyword">public</span> <span class="keyword">class </span>NonLinearSVMsDemo {</div><div class="line">    <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keywordtype">int</span> NTRAINING_SAMPLES = 100;</div><div class="line">    <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keywordtype">float</span> FRAC_LINEAR_SEP = 0.9f;</div><div class="line"></div><div class="line">    <span class="keyword">public</span> <span class="keyword">static</span> <span class="keywordtype">void</span> main(<a class="code" href="../../dc/d84/group__core__basic.html#ga1f6634802eeadfd7245bc75cf3e216c2">String</a>[] args) {</div><div class="line">        <span class="comment">// Load the native OpenCV library</span></div><div class="line">        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);</div><div class="line"></div><div class="line">        System.out.println(<span class="stringliteral">&quot;\n--------------------------------------------------------------------------&quot;</span>);</div><div class="line">        System.out.println(<span class="stringliteral">&quot;This program shows Support Vector Machines for Non-Linearly Separable Data. &quot;</span>);</div><div class="line">        System.out.println(<span class="stringliteral">&quot;--------------------------------------------------------------------------\n&quot;</span>);</div><div class="line"></div><div class="line">        <span class="comment">// Data for visual representation</span></div><div class="line">        <span class="keywordtype">int</span> width = 512, height = 512;</div><div class="line">        Mat I = Mat.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a0b57b6a326c8876d944d188a46e0f556">zeros</a>(height, width, CvType.CV_8UC3);</div><div class="line"></div><div class="line">        <span class="comment">// --------------------- 1. Set up training data randomly---------------------------------------</span></div><div class="line">        Mat trainData = <span class="keyword">new</span> Mat(2 * NTRAINING_SAMPLES, 2, CvType.CV_32F);</div><div class="line">        Mat labels = <span class="keyword">new</span> Mat(2 * NTRAINING_SAMPLES, 1, CvType.CV_32S);</div><div class="line"></div><div class="line">        Random rng = <span class="keyword">new</span> Random(100); <span class="comment">// Random value generation class</span></div><div class="line"></div><div class="line">        <span class="comment">// Set up the linearly separable part of the training data</span></div><div class="line">        <span class="keywordtype">int</span> nLinearSamples = (int) (FRAC_LINEAR_SEP * NTRAINING_SAMPLES);</div><div class="line"></div><div class="line">        <span class="comment">// Generate random points for the class 1</span></div><div class="line">        Mat trainClass = trainData.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(0, nLinearSamples);</div><div class="line">        <span class="comment">// The x coordinate of the points is in [0, 0.4)</span></div><div class="line">        Mat c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(0, 1);</div><div class="line">        <span class="keywordtype">float</span>[] cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        <span class="keywordtype">double</span>[] cDataDbl = rng.doubles(cData.length, 0, 0.4f * width).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line">        <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">        c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(1, 2);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0, height).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line"></div><div class="line">        <span class="comment">// Generate random points for the class 2</span></div><div class="line">        trainClass = trainData.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(2 * NTRAINING_SAMPLES - nLinearSamples, 2 * NTRAINING_SAMPLES);</div><div class="line">        <span class="comment">// The x coordinate of the points is in [0.6, 1]</span></div><div class="line">        c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(0, 1);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0.6 * width, width).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line">        <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">        c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(1, 2);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0, height).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line"></div><div class="line">        <span class="comment">// ------------------ Set up the non-linearly separable part of the training data ---------------</span></div><div class="line"><span class="comment"></span>        <span class="comment">// Generate random points for the classes 1 and 2</span></div><div class="line">        trainClass = trainData.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(nLinearSamples, 2 * NTRAINING_SAMPLES - nLinearSamples);</div><div class="line">        <span class="comment">// The x coordinate of the points is in [0.4, 0.6)</span></div><div class="line">        c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(0, 1);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0.4 * width, 0.6 * width).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line">        <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">        c = trainClass.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aadc8f9210fe4dec50513746c246fa8d9">colRange</a>(1, 2);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0, height).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line"></div><div class="line">        <span class="comment">// ------------------------- Set up the labels for the classes---------------------------------</span></div><div class="line">        labels.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(0, NTRAINING_SAMPLES).<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a0440e2a164c0b0d8462fb1e487be9876">setTo</a>(<span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(1)); <span class="comment">// Class 1</span></div><div class="line">        labels.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa6542193430356ad631a9beabc624107">rowRange</a>(NTRAINING_SAMPLES, 2 * NTRAINING_SAMPLES).<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a0440e2a164c0b0d8462fb1e487be9876">setTo</a>(<span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(2)); <span class="comment">// Class 2</span></div><div class="line"></div><div class="line">        <span class="comment">// ------------------------ 2. Set up the support vector machines parameters--------------------</span></div><div class="line">        System.out.println(<span class="stringliteral">&quot;Starting training process&quot;</span>);</div><div class="line">        SVM svm = SVM.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a55ced2c8d844d683ea9a725c60037ad0">create</a>();</div><div class="line">        svm.setType(SVM.C_SVC);</div><div class="line">        svm.setC(0.1);</div><div class="line">        svm.setKernel(SVM.LINEAR);</div><div class="line">        svm.setTermCriteria(<span class="keyword">new</span> TermCriteria(TermCriteria.MAX_ITER, (<span class="keywordtype">int</span>) 1e7, 1e-6));</div><div class="line"></div><div class="line">        <span class="comment">// ------------------------ 3. Train the svm----------------------------------------------------</span></div><div class="line"><span class="comment"></span>        svm.train(trainData, Ml.ROW_SAMPLE, labels);</div><div class="line">        System.out.println(<span class="stringliteral">&quot;Finished training process&quot;</span>);</div><div class="line"></div><div class="line">        <span class="comment">// ------------------------ 4. Show the decision regions----------------------------------------</span></div><div class="line"><span class="comment"></span>        byte[] IData = <span class="keyword">new</span> byte[(int) (I.total() * I.channels())];</div><div class="line">        Mat sampleMat = <span class="keyword">new</span> Mat(1, 2, CvType.CV_32F);</div><div class="line">        <span class="keywordtype">float</span>[] sampleMatData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (sampleMat.total() * sampleMat.channels())];</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; I.rows(); i++) {</div><div class="line">            <span class="keywordflow">for</span> (<span class="keywordtype">int</span> j = 0; j &lt; I.cols(); j++) {</div><div class="line">                sampleMatData[0] = j;</div><div class="line">                sampleMatData[1] = i;</div><div class="line">                sampleMat.put(0, 0, sampleMatData);</div><div class="line">                <span class="keywordtype">float</span> response = svm.predict(sampleMat);</div><div class="line"></div><div class="line">                <span class="keywordflow">if</span> (response == 1) {</div><div class="line">                    IData[(i * I.cols() + j) * I.channels()] = 0;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 1] = 100;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 2] = 0;</div><div class="line">                } <span class="keywordflow">else</span> <span class="keywordflow">if</span> (response == 2) {</div><div class="line">                    IData[(i * I.cols() + j) * I.channels()] = 100;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 1] = 0;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 2] = 0;</div><div class="line">                }</div><div class="line">            }</div><div class="line">        }</div><div class="line">        I.put(0, 0, IData);</div><div class="line"></div><div class="line">        <span class="comment">// ----------------------- 5. Show the training data--------------------------------------------</span></div><div class="line"><span class="comment"></span>        <span class="keywordtype">int</span> thick = -1;</div><div class="line">        <span class="keywordtype">int</span> lineType = Imgproc.LINE_8;</div><div class="line">        <span class="keywordtype">float</span> px, py;</div><div class="line">        <span class="comment">// Class 1</span></div><div class="line">        <span class="keywordtype">float</span>[] trainDataData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (trainData.total() * trainData.channels())];</div><div class="line">        trainData.get(0, 0, trainDataData);</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; NTRAINING_SAMPLES; i++) {</div><div class="line">            px = trainDataData[i * trainData.cols()];</div><div class="line">            py = trainDataData[i * trainData.cols() + 1];</div><div class="line">            Imgproc.circle(I, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(px, py), 3, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0, 255, 0), thick, lineType, 0);</div><div class="line">        }</div><div class="line">        <span class="comment">// Class 2</span></div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = NTRAINING_SAMPLES; i &lt; 2 * NTRAINING_SAMPLES; ++i) {</div><div class="line">            px = trainDataData[i * trainData.cols()];</div><div class="line">            py = trainDataData[i * trainData.cols() + 1];</div><div class="line">            Imgproc.circle(I, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(px, py), 3, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(255, 0, 0), thick, lineType, 0);</div><div class="line">        }</div><div class="line"></div><div class="line">        <span class="comment">// ------------------------- 6. Show support vectors--------------------------------------------</span></div><div class="line"><span class="comment"></span>        thick = 2;</div><div class="line">        Mat sv = svm.getUncompressedSupportVectors();</div><div class="line">        <span class="keywordtype">float</span>[] svData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (sv.total() * sv.channels())];</div><div class="line">        sv.get(0, 0, svData);</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; sv.rows(); i++) {</div><div class="line">            Imgproc.circle(I, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(svData[i * sv.cols()], svData[i * sv.cols() + 1]), 6, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(128, 128, 128),</div><div class="line">                    thick, lineType, 0);</div><div class="line">        }</div><div class="line"></div><div class="line">        Imgcodecs.imwrite(<span class="stringliteral">&quot;result.png&quot;</span>, I); <span class="comment">// save the Image</span></div><div class="line">        HighGui.imshow(<span class="stringliteral">&quot;SVM for Non-Linear Training Data&quot;</span>, I); <span class="comment">// show it to the user</span></div><div class="line">        HighGui.waitKey();</div><div class="line">        System.exit(0);</div><div class="line">    }</div><div class="line">}</div></div><!-- fragment -->  </div> </li>
</ul>
 <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'><ul>
<li><b>Downloadable code</b>: Click <a href="https://github.com/opencv/opencv/tree/master/samples/python/tutorial_code/ml/non_linear_svms/non_linear_svms.py">here</a></li>
<li><b>Code at glance:</b> <div class="fragment"><div class="line"><span class="keyword">from</span> __future__ <span class="keyword">import</span> print_function</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> random <span class="keyword">as</span> rng</div><div class="line"></div><div class="line">NTRAINING_SAMPLES = 100 <span class="comment"># Number of training samples per class</span></div><div class="line">FRAC_LINEAR_SEP = 0.9   <span class="comment"># Fraction of samples which compose the linear separable part</span></div><div class="line"></div><div class="line"><span class="comment"># Data for visual representation</span></div><div class="line">WIDTH = 512</div><div class="line">HEIGHT = 512</div><div class="line">I = np.zeros((HEIGHT, WIDTH, 3), dtype=np.uint8)</div><div class="line"></div><div class="line"><span class="comment"># --------------------- 1. Set up training data randomly ---------------------------------------</span></div><div class="line">trainData = np.empty((2*NTRAINING_SAMPLES, 2), dtype=np.float32)</div><div class="line">labels = np.empty((2*NTRAINING_SAMPLES, 1), dtype=np.int32)</div><div class="line"></div><div class="line">rng.seed(100) <span class="comment"># Random value generation class</span></div><div class="line"></div><div class="line"><span class="comment"># Set up the linearly separable part of the training data</span></div><div class="line">nLinearSamples = int(FRAC_LINEAR_SEP * NTRAINING_SAMPLES)</div><div class="line"></div><div class="line"></div><div class="line">trainClass = trainData[0:nLinearSamples,:]</div><div class="line"><span class="comment"># The x coordinate of the points is in [0, 0.4)</span></div><div class="line">c = trainClass[:,0:1]</div><div class="line">c[:] = np.random.uniform(0.0, 0.4 * WIDTH, c.shape)</div><div class="line"><span class="comment"># The y coordinate of the points is in [0, 1)</span></div><div class="line">c = trainClass[:,1:2]</div><div class="line">c[:] = np.random.uniform(0.0, HEIGHT, c.shape)</div><div class="line"></div><div class="line"><span class="comment"># Generate random points for the class 2</span></div><div class="line">trainClass = trainData[2*NTRAINING_SAMPLES-nLinearSamples:2*NTRAINING_SAMPLES,:]</div><div class="line"><span class="comment"># The x coordinate of the points is in [0.6, 1]</span></div><div class="line">c = trainClass[:,0:1]</div><div class="line">c[:] = np.random.uniform(0.6*WIDTH, WIDTH, c.shape)</div><div class="line"><span class="comment"># The y coordinate of the points is in [0, 1)</span></div><div class="line">c = trainClass[:,1:2]</div><div class="line">c[:] = np.random.uniform(0.0, HEIGHT, c.shape)</div><div class="line"></div><div class="line"></div><div class="line"><span class="comment">#------------------ Set up the non-linearly separable part of the training data ---------------</span></div><div class="line"></div><div class="line">trainClass = trainData[nLinearSamples:2*NTRAINING_SAMPLES-nLinearSamples,:]</div><div class="line"><span class="comment"># The x coordinate of the points is in [0.4, 0.6)</span></div><div class="line">c = trainClass[:,0:1]</div><div class="line">c[:] = np.random.uniform(0.4*WIDTH, 0.6*WIDTH, c.shape)</div><div class="line"><span class="comment"># The y coordinate of the points is in [0, 1)</span></div><div class="line">c = trainClass[:,1:2]</div><div class="line">c[:] = np.random.uniform(0.0, HEIGHT, c.shape)</div><div class="line"></div><div class="line"></div><div class="line"><span class="comment">#------------------------- Set up the labels for the classes ---------------------------------</span></div><div class="line">labels[0:NTRAINING_SAMPLES,:] = 1                   <span class="comment"># Class 1</span></div><div class="line">labels[NTRAINING_SAMPLES:2*NTRAINING_SAMPLES,:] = 2 <span class="comment"># Class 2</span></div><div class="line"></div><div class="line"><span class="comment">#------------------------ 2. Set up the support vector machines parameters --------------------</span></div><div class="line"><a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>(<span class="stringliteral">&#39;Starting training process&#39;</span>)</div><div class="line"></div><div class="line">svm = cv.ml.SVM_create()</div><div class="line">svm.setType(cv.ml.SVM_C_SVC)</div><div class="line">svm.setC(0.1)</div><div class="line">svm.setKernel(cv.ml.SVM_LINEAR)</div><div class="line">svm.setTermCriteria((cv.TERM_CRITERIA_MAX_ITER, int(1e7), 1e-6))</div><div class="line"></div><div class="line"></div><div class="line"><span class="comment">#------------------------ 3. Train the svm ----------------------------------------------------</span></div><div class="line"></div><div class="line">svm.train(trainData, cv.ml.ROW_SAMPLE, labels)</div><div class="line"></div><div class="line"><a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>(<span class="stringliteral">&#39;Finished training process&#39;</span>)</div><div class="line"></div><div class="line"><span class="comment">#------------------------ 4. Show the decision regions ----------------------------------------</span></div><div class="line"></div><div class="line">green = (0,100,0)</div><div class="line">blue = (100,0,0)</div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(I.shape[0]):</div><div class="line">    <span class="keywordflow">for</span> j <span class="keywordflow">in</span> range(I.shape[1]):</div><div class="line">        sampleMat = np.matrix([[j,i]], dtype=np.float32)</div><div class="line">        response = svm.predict(sampleMat)[1]</div><div class="line"></div><div class="line">        <span class="keywordflow">if</span> response == 1:</div><div class="line">            I[i,j] = green</div><div class="line">        <span class="keywordflow">elif</span> response == 2:</div><div class="line">            I[i,j] = blue</div><div class="line"></div><div class="line"></div><div class="line"><span class="comment">#----------------------- 5. Show the training data --------------------------------------------</span></div><div class="line"></div><div class="line">thick = -1</div><div class="line"><span class="comment"># Class 1</span></div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(NTRAINING_SAMPLES):</div><div class="line">    px = trainData[i,0]</div><div class="line">    py = trainData[i,1]</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">cv.circle</a>(I, (int(px), int(py)), 3, (0, 255, 0), thick)</div><div class="line"></div><div class="line"><span class="comment"># Class 2</span></div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(NTRAINING_SAMPLES, 2*NTRAINING_SAMPLES):</div><div class="line">    px = trainData[i,0]</div><div class="line">    py = trainData[i,1]</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">cv.circle</a>(I, (int(px), int(py)), 3, (255, 0, 0), thick)</div><div class="line"></div><div class="line"></div><div class="line"><span class="comment">#------------------------- 6. Show support vectors --------------------------------------------</span></div><div class="line"></div><div class="line">thick = 2</div><div class="line">sv = svm.getUncompressedSupportVectors()</div><div class="line"></div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(sv.shape[0]):</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">cv.circle</a>(I, (int(sv[i,0]), int(sv[i,1])), 6, (128, 128, 128), thick)</div><div class="line"></div><div class="line"></div><div class="line"><a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv.imwrite</a>(<span class="stringliteral">&#39;result.png&#39;</span>, I)                      <span class="comment"># save the Image</span></div><div class="line"><a class="code" href="../../df/d24/group__highgui__opengl.html#gaae7e90aa3415c68dba22a5ff2cefc25d">cv.imshow</a>(<span class="stringliteral">&#39;SVM for Non-Linear Training Data&#39;</span>, I) <span class="comment"># show it to the user</span></div><div class="line"><a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">cv.waitKey</a>()</div></div><!-- fragment -->  </div> </li>
</ul>
<h2>Explanation </h2>
<ul>
<li><b>Set up the training data</b></li>
</ul>
<p>The training data of this exercise is formed by a set of labeled 2D-points that belong to one of two different classes. To make the exercise more appealing, the training data is generated randomly using a uniform probability density functions (PDFs).</p>
<p>We have divided the generation of the training data into two main parts.</p>
<p>In the first part we generate data for both classes that is linearly separable.</p>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'> <div class="fragment"><div class="line">    <span class="comment">// Generate random points for the class 1</span></div><div class="line">    Mat trainClass = trainData.rowRange(0, nLinearSamples);</div><div class="line">    <span class="comment">// The x coordinate of the points is in [0, 0.4)</span></div><div class="line">    Mat c = trainClass.colRange(0, 1);</div><div class="line">    rng.fill(c, RNG::UNIFORM, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.4 * WIDTH));</div><div class="line">    <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">    c = trainClass.colRange(1,2);</div><div class="line">    rng.fill(c, RNG::UNIFORM, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HEIGHT));</div><div class="line"></div><div class="line">    <span class="comment">// Generate random points for the class 2</span></div><div class="line">    trainClass = trainData.rowRange(2*NTRAINING_SAMPLES-nLinearSamples, 2*NTRAINING_SAMPLES);</div><div class="line">    <span class="comment">// The x coordinate of the points is in [0.6, 1]</span></div><div class="line">    c = trainClass.colRange(0 , 1);</div><div class="line">    rng.fill(c, RNG::UNIFORM, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.6*WIDTH), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(WIDTH));</div><div class="line">    <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">    c = trainClass.colRange(1,2);</div><div class="line">    rng.fill(c, RNG::UNIFORM, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HEIGHT));</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'> <div class="fragment"><div class="line">        <span class="comment">// Generate random points for the class 1</span></div><div class="line">        Mat trainClass = trainData.rowRange(0, nLinearSamples);</div><div class="line">        <span class="comment">// The x coordinate of the points is in [0, 0.4)</span></div><div class="line">        Mat c = trainClass.colRange(0, 1);</div><div class="line">        <span class="keywordtype">float</span>[] cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        <span class="keywordtype">double</span>[] cDataDbl = rng.doubles(cData.length, 0, 0.4f * width).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line">        <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">        c = trainClass.colRange(1, 2);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0, height).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line"></div><div class="line">        <span class="comment">// Generate random points for the class 2</span></div><div class="line">        trainClass = trainData.rowRange(2 * NTRAINING_SAMPLES - nLinearSamples, 2 * NTRAINING_SAMPLES);</div><div class="line">        <span class="comment">// The x coordinate of the points is in [0.6, 1]</span></div><div class="line">        c = trainClass.colRange(0, 1);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0.6 * width, width).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line">        <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">        c = trainClass.colRange(1, 2);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0, height).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'> <div class="fragment"><div class="line"><span class="comment"># Generate random points for the class 1</span></div><div class="line">trainClass = trainData[0:nLinearSamples,:]</div><div class="line"><span class="comment"># The x coordinate of the points is in [0, 0.4)</span></div><div class="line">c = trainClass[:,0:1]</div><div class="line">c[:] = np.random.uniform(0.0, 0.4 * WIDTH, c.shape)</div><div class="line"><span class="comment"># The y coordinate of the points is in [0, 1)</span></div><div class="line">c = trainClass[:,1:2]</div><div class="line">c[:] = np.random.uniform(0.0, HEIGHT, c.shape)</div><div class="line"></div><div class="line"><span class="comment"># Generate random points for the class 2</span></div><div class="line">trainClass = trainData[2*NTRAINING_SAMPLES-nLinearSamples:2*NTRAINING_SAMPLES,:]</div><div class="line"><span class="comment"># The x coordinate of the points is in [0.6, 1]</span></div><div class="line">c = trainClass[:,0:1]</div><div class="line">c[:] = np.random.uniform(0.6*WIDTH, WIDTH, c.shape)</div><div class="line"><span class="comment"># The y coordinate of the points is in [0, 1)</span></div><div class="line">c = trainClass[:,1:2]</div><div class="line">c[:] = np.random.uniform(0.0, HEIGHT, c.shape)</div></div><!-- fragment --> </div> <p>In the second part we create data for both classes that is non-linearly separable, data that overlaps.</p>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'> <div class="fragment"><div class="line">    <span class="comment">// Generate random points for the classes 1 and 2</span></div><div class="line">    trainClass = trainData.rowRange(nLinearSamples, 2*NTRAINING_SAMPLES-nLinearSamples);</div><div class="line">    <span class="comment">// The x coordinate of the points is in [0.4, 0.6)</span></div><div class="line">    c = trainClass.colRange(0,1);</div><div class="line">    rng.fill(c, RNG::UNIFORM, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.4*WIDTH), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0.6*WIDTH));</div><div class="line">    <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">    c = trainClass.colRange(1,2);</div><div class="line">    rng.fill(c, RNG::UNIFORM, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HEIGHT));</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'> <div class="fragment"><div class="line">        <span class="comment">// Generate random points for the classes 1 and 2</span></div><div class="line">        trainClass = trainData.rowRange(nLinearSamples, 2 * NTRAINING_SAMPLES - nLinearSamples);</div><div class="line">        <span class="comment">// The x coordinate of the points is in [0.4, 0.6)</span></div><div class="line">        c = trainClass.colRange(0, 1);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0.4 * width, 0.6 * width).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div><div class="line">        <span class="comment">// The y coordinate of the points is in [0, 1)</span></div><div class="line">        c = trainClass.colRange(1, 2);</div><div class="line">        cData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (c.total() * c.channels())];</div><div class="line">        cDataDbl = rng.doubles(cData.length, 0, height).toArray();</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; cData.length; i++) {</div><div class="line">            cData[i] = (float) cDataDbl[i];</div><div class="line">        }</div><div class="line">        c.put(0, 0, cData);</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'> <div class="fragment"><div class="line"><span class="comment"># Generate random points for the classes 1 and 2</span></div><div class="line">trainClass = trainData[nLinearSamples:2*NTRAINING_SAMPLES-nLinearSamples,:]</div><div class="line"><span class="comment"># The x coordinate of the points is in [0.4, 0.6)</span></div><div class="line">c = trainClass[:,0:1]</div><div class="line">c[:] = np.random.uniform(0.4*WIDTH, 0.6*WIDTH, c.shape)</div><div class="line"><span class="comment"># The y coordinate of the points is in [0, 1)</span></div><div class="line">c = trainClass[:,1:2]</div><div class="line">c[:] = np.random.uniform(0.0, HEIGHT, c.shape)</div></div><!-- fragment --> </div> <ul>
<li><b>Set up SVM's parameters</b></li>
</ul>
<dl class="section note"><dt>Note</dt><dd>In the previous tutorial <a class="el" href="../../d1/d73/tutorial_introduction_to_svm.html">Introduction to Support Vector Machines</a> there is an explanation of the attributes of the class <a class="el" href="../../d1/d2d/classcv_1_1ml_1_1SVM.html">cv::ml::SVM</a> that we configure here before training the SVM.</dd></dl>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'> <div class="fragment"><div class="line">    Ptr&lt;SVM&gt; svm = SVM::create();</div><div class="line">    svm-&gt;setType(SVM::C_SVC);</div><div class="line">    svm-&gt;setC(0.1);</div><div class="line">    svm-&gt;setKernel(<a class="code" href="../../df/d5b/group__fuzzy.html#gga847dcdc873389a919e2c3ba910a2bd39a4d3d0c977941e929e23e9c6cb4ae4715">SVM::LINEAR</a>);</div><div class="line">    svm-&gt;setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, (<span class="keywordtype">int</span>)1e7, 1e-6));</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'> <div class="fragment"><div class="line">        SVM svm = SVM.create();</div><div class="line">        svm.setType(SVM.C_SVC);</div><div class="line">        svm.setC(0.1);</div><div class="line">        svm.setKernel(SVM.LINEAR);</div><div class="line">        svm.setTermCriteria(<span class="keyword">new</span> TermCriteria(TermCriteria.MAX_ITER, (<span class="keywordtype">int</span>) 1e7, 1e-6));</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'> <div class="fragment"><div class="line">svm = cv.ml.SVM_create()</div><div class="line">svm.setType(cv.ml.SVM_C_SVC)</div><div class="line">svm.setC(0.1)</div><div class="line">svm.setKernel(cv.ml.SVM_LINEAR)</div><div class="line">svm.setTermCriteria((cv.TERM_CRITERIA_MAX_ITER, int(1e7), 1e-6))</div></div><!-- fragment --> </div> <p>There are just two differences between the configuration we do here and the one that was done in the previous tutorial (<a class="el" href="../../d1/d73/tutorial_introduction_to_svm.html">Introduction to Support Vector Machines</a>) that we use as reference.</p>
<ul>
<li><p class="startli"><em>C</em>. We chose here a small value of this parameter in order not to punish too much the misclassification errors in the optimization. The idea of doing this stems from the will of obtaining a solution close to the one intuitively expected. However, we recommend to get a better insight of the problem by making adjustments to this parameter.</p>
<dl class="section note"><dt>Note</dt><dd>In this case there are just very few points in the overlapping region between classes. By giving a smaller value to <b>FRAC_LINEAR_SEP</b> the density of points can be incremented and the impact of the parameter <em>C</em> explored deeply.</dd></dl>
</li>
<li><em>Termination Criteria of the algorithm</em>. The maximum number of iterations has to be increased considerably in order to solve correctly a problem with non-linearly separable training data. In particular, we have increased in five orders of magnitude this value.</li>
<li><b>Train the SVM</b></li>
</ul>
<p>We call the method <a class="el" href="../../db/d7d/classcv_1_1ml_1_1StatModel.html#af96a0e04f1677a835cc25263c7db3c0c">cv::ml::SVM::train</a> to build the SVM model. Watch out that the training process may take a quite long time. Have patiance when your run the program.</p>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'> <div class="fragment"><div class="line">    svm-&gt;train(trainData, <a class="code" href="../../dd/ded/group__ml.html#gga9c57a2b823008dda53d2c7f7059a8710ab8565ac2eb79152a4e3c80b0e9c9fd4c">ROW_SAMPLE</a>, labels);</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'> <div class="fragment"><div class="line">        svm.train(trainData, Ml.ROW_SAMPLE, labels);</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'> <div class="fragment"><div class="line">svm.train(trainData, cv.ml.ROW_SAMPLE, labels)</div></div><!-- fragment --> </div> <ul>
<li><b>Show the Decision Regions</b></li>
</ul>
<p>The method <a class="el" href="../../db/d7d/classcv_1_1ml_1_1StatModel.html#a1a7e49e1febd10392452727498771bc1">cv::ml::SVM::predict</a> is used to classify an input sample using a trained SVM. In this example we have used this method in order to color the space depending on the prediction done by the SVM. In other words, an image is traversed interpreting its pixels as points of the Cartesian plane. Each of the points is colored depending on the class predicted by the SVM; in dark green if it is the class with label 1 and in dark blue if it is the class with label 2.</p>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'> <div class="fragment"><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga7e6060c0b8d48459964df6e1eb524c03">Vec3b</a> green(0,100,0), blue(100,0,0);</div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; I.rows; i++)</div><div class="line">    {</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> j = 0; j &lt; I.cols; j++)</div><div class="line">        {</div><div class="line">            Mat sampleMat = (Mat_&lt;float&gt;(1,2) &lt;&lt; j, i);</div><div class="line">            <span class="keywordtype">float</span> response = svm-&gt;predict(sampleMat);</div><div class="line"></div><div class="line">            <span class="keywordflow">if</span>      (response == 1) I.at&lt;<a class="code" href="../../dc/d84/group__core__basic.html#ga7e6060c0b8d48459964df6e1eb524c03">Vec3b</a>&gt;(i,j) = green;</div><div class="line">            <span class="keywordflow">else</span> <span class="keywordflow">if</span> (response == 2) I.at&lt;<a class="code" href="../../dc/d84/group__core__basic.html#ga7e6060c0b8d48459964df6e1eb524c03">Vec3b</a>&gt;(i,j) = blue;</div><div class="line">        }</div><div class="line">    }</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'> <div class="fragment"><div class="line">        byte[] IData = <span class="keyword">new</span> byte[(int) (I.total() * I.channels())];</div><div class="line">        Mat sampleMat = <span class="keyword">new</span> Mat(1, 2, CvType.CV_32F);</div><div class="line">        <span class="keywordtype">float</span>[] sampleMatData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (sampleMat.total() * sampleMat.channels())];</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; I.rows(); i++) {</div><div class="line">            <span class="keywordflow">for</span> (<span class="keywordtype">int</span> j = 0; j &lt; I.cols(); j++) {</div><div class="line">                sampleMatData[0] = j;</div><div class="line">                sampleMatData[1] = i;</div><div class="line">                sampleMat.put(0, 0, sampleMatData);</div><div class="line">                <span class="keywordtype">float</span> response = svm.predict(sampleMat);</div><div class="line"></div><div class="line">                <span class="keywordflow">if</span> (response == 1) {</div><div class="line">                    IData[(i * I.cols() + j) * I.channels()] = 0;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 1] = 100;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 2] = 0;</div><div class="line">                } <span class="keywordflow">else</span> <span class="keywordflow">if</span> (response == 2) {</div><div class="line">                    IData[(i * I.cols() + j) * I.channels()] = 100;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 1] = 0;</div><div class="line">                    IData[(i * I.cols() + j) * I.channels() + 2] = 0;</div><div class="line">                }</div><div class="line">            }</div><div class="line">        }</div><div class="line">        I.put(0, 0, IData);</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'> <div class="fragment"><div class="line">green = (0,100,0)</div><div class="line">blue = (100,0,0)</div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(I.shape[0]):</div><div class="line">    <span class="keywordflow">for</span> j <span class="keywordflow">in</span> range(I.shape[1]):</div><div class="line">        sampleMat = np.matrix([[j,i]], dtype=np.float32)</div><div class="line">        response = svm.predict(sampleMat)[1]</div><div class="line"></div><div class="line">        <span class="keywordflow">if</span> response == 1:</div><div class="line">            I[i,j] = green</div><div class="line">        <span class="keywordflow">elif</span> response == 2:</div><div class="line">            I[i,j] = blue</div></div><!-- fragment --> </div> <ul>
<li><b>Show the training data</b></li>
</ul>
<p>The method <a class="el" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">cv::circle</a> is used to show the samples that compose the training data. The samples of the class labeled with 1 are shown in light green and in light blue the samples of the class labeled with 2.</p>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'> <div class="fragment"><div class="line">    <span class="keywordtype">int</span> thick = -1;</div><div class="line">    <span class="keywordtype">float</span> px, py;</div><div class="line">    <span class="comment">// Class 1</span></div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; NTRAINING_SAMPLES; i++)</div><div class="line">    {</div><div class="line">        px = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,0);</div><div class="line">        py = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,1);</div><div class="line">        <a class="code" href="../../d9/db7/group__datasets__gr.html#gga610754124ced68d1f05760b5948fbb76a6f0d8b2d9e3e947b2a5c1eff9e81ee95">circle</a>(I, <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>( (<span class="keywordtype">int</span>) px,  (<span class="keywordtype">int</span>) py ), 3, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0, 255, 0), thick);</div><div class="line">    }</div><div class="line">    <span class="comment">// Class 2</span></div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = NTRAINING_SAMPLES; i &lt;2*NTRAINING_SAMPLES; i++)</div><div class="line">    {</div><div class="line">        px = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,0);</div><div class="line">        py = trainData.at&lt;<span class="keywordtype">float</span>&gt;(i,1);</div><div class="line">        <a class="code" href="../../d9/db7/group__datasets__gr.html#gga610754124ced68d1f05760b5948fbb76a6f0d8b2d9e3e947b2a5c1eff9e81ee95">circle</a>(I, <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>( (<span class="keywordtype">int</span>) px, (<span class="keywordtype">int</span>) py ), 3, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(255, 0, 0), thick);</div><div class="line">    }</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'> <div class="fragment"><div class="line">        <span class="keywordtype">int</span> thick = -1;</div><div class="line">        <span class="keywordtype">int</span> lineType = Imgproc.LINE_8;</div><div class="line">        <span class="keywordtype">float</span> px, py;</div><div class="line">        <span class="comment">// Class 1</span></div><div class="line">        <span class="keywordtype">float</span>[] trainDataData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (trainData.total() * trainData.channels())];</div><div class="line">        trainData.get(0, 0, trainDataData);</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; NTRAINING_SAMPLES; i++) {</div><div class="line">            px = trainDataData[i * trainData.cols()];</div><div class="line">            py = trainDataData[i * trainData.cols() + 1];</div><div class="line">            Imgproc.circle(I, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(px, py), 3, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(0, 255, 0), thick, lineType, 0);</div><div class="line">        }</div><div class="line">        <span class="comment">// Class 2</span></div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = NTRAINING_SAMPLES; i &lt; 2 * NTRAINING_SAMPLES; ++i) {</div><div class="line">            px = trainDataData[i * trainData.cols()];</div><div class="line">            py = trainDataData[i * trainData.cols() + 1];</div><div class="line">            Imgproc.circle(I, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(px, py), 3, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(255, 0, 0), thick, lineType, 0);</div><div class="line">        }</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'> <div class="fragment"><div class="line">thick = -1</div><div class="line"><span class="comment"># Class 1</span></div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(NTRAINING_SAMPLES):</div><div class="line">    px = trainData[i,0]</div><div class="line">    py = trainData[i,1]</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">cv.circle</a>(I, (int(px), int(py)), 3, (0, 255, 0), thick)</div><div class="line"></div><div class="line"><span class="comment"># Class 2</span></div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(NTRAINING_SAMPLES, 2*NTRAINING_SAMPLES):</div><div class="line">    px = trainData[i,0]</div><div class="line">    py = trainData[i,1]</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">cv.circle</a>(I, (int(px), int(py)), 3, (255, 0, 0), thick)</div></div><!-- fragment --> </div> <ul>
<li><b>Support vectors</b></li>
</ul>
<p>We use here a couple of methods to obtain information about the support vectors. The method <a class="el" href="../../d1/d2d/classcv_1_1ml_1_1SVM.html#a2c3fb4b3c80b8fce0b8654f103339300">cv::ml::SVM::getSupportVectors</a> obtain all support vectors. We have used this methods here to find the training examples that are support vectors and highlight them.</p>
 <div class='newInnerHTML' title='cpp' style='display: none;'>C++</div><div class='toggleable_div label_cpp' style='display: none;'> <div class="fragment"><div class="line">    thick = 2;</div><div class="line">    Mat sv = svm-&gt;getUncompressedSupportVectors();</div><div class="line"></div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; sv.rows; i++)</div><div class="line">    {</div><div class="line">        <span class="keyword">const</span> <span class="keywordtype">float</span>* v = sv.ptr&lt;<span class="keywordtype">float</span>&gt;(i);</div><div class="line">        <a class="code" href="../../d9/db7/group__datasets__gr.html#gga610754124ced68d1f05760b5948fbb76a6f0d8b2d9e3e947b2a5c1eff9e81ee95">circle</a>(I,  <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>( (<span class="keywordtype">int</span>) v[0], (<span class="keywordtype">int</span>) v[1]), 6, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(128, 128, 128), thick);</div><div class="line">    }</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='java' style='display: none;'>Java</div><div class='toggleable_div label_java' style='display: none;'> <div class="fragment"><div class="line">        thick = 2;</div><div class="line">        Mat sv = svm.getUncompressedSupportVectors();</div><div class="line">        <span class="keywordtype">float</span>[] svData = <span class="keyword">new</span> <span class="keywordtype">float</span>[(int) (sv.total() * sv.channels())];</div><div class="line">        sv.get(0, 0, svData);</div><div class="line">        <span class="keywordflow">for</span> (<span class="keywordtype">int</span> i = 0; i &lt; sv.rows(); i++) {</div><div class="line">            Imgproc.circle(I, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(svData[i * sv.cols()], svData[i * sv.cols() + 1]), 6, <span class="keyword">new</span> <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(128, 128, 128),</div><div class="line">                    thick, lineType, 0);</div><div class="line">        }</div></div><!-- fragment --> </div>  <div class='newInnerHTML' title='python' style='display: none;'>Python</div><div class='toggleable_div label_python' style='display: none;'> <div class="fragment"><div class="line">thick = 2</div><div class="line">sv = svm.getUncompressedSupportVectors()</div><div class="line"></div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(sv.shape[0]):</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gaf10604b069374903dbd0f0488cb43670">cv.circle</a>(I, (int(sv[i,0]), int(sv[i,1])), 6, (128, 128, 128), thick)</div></div><!-- fragment --> </div> <h2>Results </h2>
<ul>
<li>The code opens an image and shows the training examples of both classes. The points of one class are represented with light green and light blue ones are used for the other class.</li>
<li>The SVM is trained and used to classify all the pixels of the image. This results in a division of the image in a blue region and a green region. The boundary between both regions is the separating hyperplane. Since the training data is non-linearly separable, it can be seen that some of the examples of both classes are misclassified; some green points lay on the blue region and some blue points lay on the green one.</li>
<li>Finally the support vectors are shown using gray rings around the training examples.</li>
</ul>
<div class="image">
<img src="../../svm_non_linear_result.png" alt="svm_non_linear_result.png"/>
</div>
<p>You may observe a runtime instance of this on the <a href="https://www.youtube.com/watch?v=vFv2yPcSo-Q">YouTube here</a>.</p>
<div align='center'><iframe title='Video' width='560' height='349' src='https://www.youtube.com/embed/vFv2yPcSo-Q?rel=0' frameborder='0' align='middle' allowfullscreen></iframe></div> </div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
